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ABSTRACT 



The initiatives taken by one school district to support 
professional development for principals in using data for school planning and 
reform were studied. A data-planning training model was developed in the San 
Francisco Unified School District (SFUSD) (California) to build skills 
principals need to read, interpret, and use multiple sources of data for 
school reform and continuous school improvement. The model was implemented 
with 17 middle school administrators in the SFUSD, and these administrators 
had the responsibility for teaching it to their staffs. Eleven of these 
principals responded to a survey about the model and the training, including 
workshops. Results of the survey suggest that administrators were actively 
using the data to plan and improve instruction. They seemed to understand the 
types of indicators to use when assessing achievement and improvements and 
they were able to read and use the accountability reports to find relevant 
information. Because middle school administrators found the data planning 
model helpful, it was extended to the elementary school level the following 
year. Three attachments illustrate the "data puzzle," a planning chart, and 
the use of the model to make a rubric for school portfolios. (Contains 2 
tables, 2 figures, and 13 references.) (SLD) 
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Introduction 



Understanding and Use of Data by Teacher sand Administrators 

Reseafchers have learned much in recent years about how teachers view ^d use resuUs 

from standardized tests of student achievement. For example, we Imow that teachers u^^^^ 

data very rarely to inform educational practice (Beck & Stetz, 1989 ; Sdmon-Cox, 198 ). , 

while teachers report they feel pressure to improve test scores, they believe such score 
particularlv usefbl in helping to drive instruction in a positive way. In addition, teachers may rmt 
Lderstand that improved test scores may not necessarily be attnbutable to 
teaching but to the employment of focused test preparation strategi^(Linn, Graue, & S^P^rs 
1989) or to changes in the demographic profile of students tested. They inay not understand k y 
terms like "Scale Score” or Normal Curve Equivalent. Nor do many teachers understand 

statistical concepts that inform the interpretation of data. 

Research on how principals understand and use such data is much less understood, and 
few studies have systematically examined how principals use such data m decision-m^ng 
Kennedv 1984- Penuel. et. al., 1998). Comparative studies have found that in general, 
principa^^ knowledge of testing and assessment is somewhat higher than teachers knowledge 
(^Impara etal., 1993). At the same time, principals have histoncally not appeared to g 
pSarly s^ong emphasis to the analysis of data. Glasman (1984) ^ 

nominated bv supervisors as the most effective in their distnct did not differ from pnncipals 
nominated as the least effective in the extent to which they use such data either to improv 

student achievement or evaluate teacher performance. <=v<;teins 

As school districts across the country develop more structured accountability systems 
with high-stakes assessment tools driving such systems, there is increasmg pressure on P^cipals 
"iStand and use such data to improve insuucuon in hte sehod. 

such strong external accountability systems is by no means given (Corbett Wilson, ) 
there are no guarantees that improvements in instruction will result in higher test scores on norm- 
r^erenced smdardized tests, principals^ jobs increasingly depend on showing 
measures and on representing and interpreting those gams favorably to the broader 

community and to district supervisors (Beck Murphy, 1996). . , . otionr,! 

oSy recently have some broad expected competencies been ^ 

administratL with respect to the use of assessment data. Impara and Flake (1996) identify the 
following tasks for administrators should be able to do with respect to the use of data: 

1. using scores to decide on special program eligibility, 

2. evaluating the school or system assessment program; 

3. monitor student performance; 

4 interpreting test scores for others; 

5 developing curriculum that matches student performance to leammg objectives, 

6. develop intervention procedures to identify student and teacher strengths and 

Am^n^rsldlls Impara and Blake (1996) list as critical to perfo^ng 
knowledge of terminology of standardized test reports, such as concepts like ’'^kability and 

ment k^^^^ of how to reconcile conflicting assessment informauon about studei^te 
Siowledge’of measurement theory; and an ability to apply assessment/measurement m practice. 
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Given the current political and educational contexts faced by principals, it is important that 
researchers and administrators begin to work collaboratively to define and build competencies in 
the area of using assessment data (Cousins & Earl, 1992). This is important as there is evidence 
that in districts with strong accountability systems that readily use data such as test scores to plan 
and implement reform strategies, principals’ willingness and ability to interpret and use data 
effectively is much greater (Penuel et. al, 1998). At the same time, this same research has 
identified critical gaps in training, namely, the need to help administrators test their own theories 
and strategies for school reform against evidence. 

Current Research Objectives c ■ \ 

In this paper, we explore one district’s initiatives to support principal professional 
development for using data for school planning and reform. The goals of this paper are two- 
fold: (1) to describe a data-planning training model developed in the San Francisco Unified 
School District (SFUSD) which was designed to build the skills needed to read, interpret, and 
effectively use multiple sources of data for the purposes of school reform and continuous school 
improvement; and (2) describe the benefits of a data-planning model for supporting 
administrators in the planning and implementation process used to address academic 
achievement. 

Accountability at San Francisco Unified School District 

Accountability System . . 

The Department of Program Evaluation and Research developed a data-planning training 
model to help support and strengthen the district’s evolving accountability system. The primary 
objective of the accountability system at the SFUSD is to support data-driven decision making 
by school administrators and teachers to improve teaching and learning throughout the district. 
Underlying this objective is a belief that by analyzing student achievement data and 
collaborativeh reflecting on data, school administrators and teachers can help schools meet the 
“learning imperative” to become successful schools that reach all children (Beck & Murphy, 

1 996). San Francisco’s data-planning model begins with a process of analyzing current 
conditions at the school; that is, gaining a better understanding of how students are performing 
now and what their specific learning needs are. Schools then develop a School Site Plan for 
meeting those learning needs, implement those plans, and monitor their progress throughout the 
year. At the end of the year, each school participates in a district-wide process of evaluating 
their progress on site-specific goals and objectives and on meeting the district s goals and 
Superintendent’s Priorities. 

While many districts throughout the United States are moving toward using data to 
support decision-making, San Francisco’s accountability plan is umque in the extent of training 
and support that is provided to administrators in this model and in the ways that data are 
analyzed. Principals throughout the district have all received training in understanding student 
test score da ta and each year develop a plan for their school based on previous’ year s student 
achievement data. They identify specific gaps not only by subject area (such as reading or math) 
or by grade but also gaps in performance by ethnicity, grade, and program. 

Supporting this data-driven decision making model is a comprehensive accounmbility 
system involving all stakeholders in the community. SFUSD’s accountability system includes a 
comprehensive assessment program to measure student learning, a technology infrastructure to 
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support data collection and analysis, strategies for recognizing success and solving problems, and 
a training plan to show administrators and teachers how to use data to guide their planning, 
instruction, and organization. 

Documents that Support Accountability 

A major aspect of the accountability system is the production of resource documents 
available to every administrator. These documents provide information regarding student 
achievement indicators. These School Accountability Reports provide schools with 
disaggregated test score data to support decision-making. The Reports are used by different 
people in the district for different purposes. For a parent, the Reports might be used to track their 
school’s performance or select a school for their child. For a district administrator, the profiles 
may be used to target assistance low-performing schools. For principals, the reports may be used 
to set school site priorities. 

Academic Achievement Volumes present the results from the district-wide administration 
of the Comprehensive Tests of Basic Skills (CTBS). Using the Normal Curve Equivalent unit of 
measure, the report presents the mean performance scores and the mean gain scores of groups of 
students who have been in the school district for at least one school year i.e. they have test results 
from both the current year and the previous year. These results are disaggregated by grade, 
ethnicity and program (e.g. Special Education, Bilingual, Title I etc.). 

The district also produces a document that measures performance of schools towards the 
District’s six goals measured under 17 objectives called the Goals and Objectives report. 
Advancement towards the qualitative objectives on this report were measured through the 
production of a school portfolio and quantitative objectives were measured through achievement 
and other indicators. A research based report titled Longitudinal Study of Attrition, Retention 
and Test Score Growth is prepared pursuant to a Court order. The report is a longitudinal smdy 
following students over a period of four years to determine if they leave the District, drop out, 
change schools or are retained. The report also examines student performance on the 
standardized test over this period. The Comprehensive School Improvement Program (CSIP) 
uses data from the Goals & Objectives report and the Longitudinal study along with other 
indicators suggested by the State and the Court to help identify low performing schools. 

The Pupil Services Department reports data on Suspensions and Expulsions. These data 
aredisaggre gated by ethnicity and offense type. Results are summarized in tabular and graphic 
form. The Health Services department administers a Youth Risk Behavior Survey every two 
years to middle and high school students. Departments share their results with administrators 
and teachers at professional development meetings. 

Information Systems To Support Accountability 

To support the production of these accountability documents, SFUSD has an extensive 
technological infrastructure that stores detailed records on students and staff. These records are 
in electronic form and can be accessed through a mainframe at the district’s downtown offices. 
The detail and accessibility of these records allows the Department of Research, Planning, and 
Evaluation to produce “data-on-demand” for progr^ administrators, principals, and central 
office administrators to support their decision-making. 

Some special features of the information system include: 

• Data for Multiple Measures of Student Achievement and Behavior: SFUSD maintains 
records not only for the CTBS/SAT-9, the district-wide assessment of student achievement. 



Supporting Data Use Among Administrators 



6 BESTCOPYAVAILABLE 

5 Khanna, R., Trousdale, D., Penuel,B. Kell, J. 



but also for the SABE (a Spanish-language test of student achievement), GPA, the Brigance 
(pre-K. K screening), and proficiency tests. For high schools, data are maintained on 
enrollment in algebra classes, honors classes. A-F classes, and AP classes. SAT and PSAT 
scores are also kept for each student. The district also maintains behavioral records for 
students, including attendance, suspensions, expulsions, retentions, and dropouts. 

• Disaggregation; Disaggregating data is one of the primary functions of the Research. 
Planning, and Evaluation departments annual accountability reporting. Because detailed 
demographic data, including gender, ethnicity, free lunch status, LEP status, and home 
language are all kept in the central database, student achievement and behavioral data can be 
analyzed by breaking down scores by any of these variables. Such disaggregation is critical 
if schools are to target interventions to students who need the most help. 

• Comparability: The district maintains records for several years for students, and longitudinal 
reports of two- and three-year trends are produced annually to track the long term growth of 
students. In these reports, those students for whom there are multiple years of records are 
tracked for their academic progress. The format of the annual reports is the same each year, 
allowing one to analyze changes in long-term trends. 

• Advanced Statistics: The district’s accountability reports do more than report descriptive 
statistics regarding test scores. The Research, Planning, and Evaluation Department has the 
capability to perform statistical tests to determine whether the changes over time are 
statistically significant. In addition, for program evaluation reports, the detailed demographic 
information available allows researchers to create matched comparison groups to measure 
more accurately the impact of programs. 

Implementing a Data-Planning Training Model in San Francisco 

Background & Objectives 

Despite the efforts of administrative credential programs which attempt to provide some 
training in data analysis and statistics, the actual use of accountability reports as decision-making 
tools by district administrators remains limited due to the lack of understanding of these complex 
documents. Further, budgetary limits in districts leave little or no funds for further training in 
understanding and planning with the data. Therefore, most of the accountability reports that are 
produced by the district remain unused by their intended audience. 

This dilemma became particularly salient among the administrators at SFUSD. Even 
though the District’s Research. Planning and Evaluation Department was producing many 
sophisticated reports with summarized statistical information on multiple measures, they were 
not being used effectively as decision-making tools. Administrators found these reports to be 
cumbersome volumes of cryptic information that were difficult to understand, let alone useful 
models for curriculum planning. 

The District developed a plan for more effective commumcation and training regarding 
the use of data for data driven decision-making which began with initial training sessioi^ that 
instructed administrators how to use read, understand, and utilize the data. The District s 
emphasis on data driven decision-making made it a key objective and as a common expectation 

of all schools. 
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Methodology , . , j , • 

In order to insure that administrators had the skills needed to design the data-planning 
ponion of their school portfolio plans, a three-year data-planning model was designed in 1995-96 
to assist all administrators with their understanding of how to reason and plan with data. The 
model employs a five step process, the first step of which is introduced in this training module; 



Analyze 

examining trends in test data, adding other sources of data to 
complete a portrait of the school’s performance; explaining 
the data; identifying other data to back explanations 



Plan 

articulating the particular reform strategies being tried in the 
school; identifying the research or practice basis for these 

strategies 



Implement 

identifying what is to be done when and by whom 



Monitor 

identifying ways to document whether what was planned 
was actually implemented; setting benchmarks 



Evaluate 

testing assumptions and theories of reform against the 

evidence 



This model was then implemented with 17 middle school administrators at SFUSD. The 
administrators then had the responsibility to teach it to their staff The key elements of the model 
involved teaching administrators about disaggregating and summarizing data before analyzin g 
f jpta and developing an action plan for the school based on findings. 

First, admini strators were introduced to the extent of information available in Ae 
accountability reports which range from profiles filled with demographics to longitudinal studies 
tracking students over four years in terms of their mobility and academic performance on a 
standardized test. Next, the training included a discussion on the usefulness of disaggregation of 
the data in these accountability reports. These accountability reports disaggregate the 
information in various ways; by program, ethnicity, grade, gender, English proficiency, socio- 
economic stams, etc. Such a breakdown of information is helpful for more specific 
understanding of strengths and weaknesses. For instance, through understanding how to analyze 
disaggregated data, middle school administrators were able to become more precise in stating a 
school priority. Administrators were also encouraged to collect their own data (quantitative or 
qualitative) related to parental involvement, school safety, school disciplirie, etc. Participants m 
the tr ainin g were given tips on data collection methods that included keeping records of data 
through incident reports, classroom observations, opinion surveys, structured interviews, and 
longitudinal growth of the school. Administrators were further encouraged to compare their data 
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to that of a demographically equivalent school similar in ethnic distribution, school size and 
budget. 

Last, the middle school administrators were instructed on how to create an action 
planning model. This involves reading and interpreting data, writing out measurable objectives, 
addressing research based reform strategies as activities and lastly, determining school pnonties 

based on the information gathered from the data analysis. f „ . 

The key points that were emphasized as administrators analyzed their data were as lollows. 

(a) It is important to consider multiple sources of data when planning, standardized test scores 

alone do not provide the basis for adequate planning effort 

(b) Data planning involves fitting different pieces of data into a puzzle telling a complete story 

of what is going on in one's school. 

(c) It is important to remember that some data are contradictory. For example, while test scores 
my have risen, attendance my have dropped. Part of the role of an administrator is to make 
sense of these contradictions by giving a complete account of the data. The data do not speak 
for themselves. Administrators give it life and meaning by the story they tell about the data. 

(d) Data analysis in this involves generating explanations for why one see patterns in the data. 
Data patterns are like the gist of the story. It is a summary from a vanety of sources of data 
that captures what the data are about. 

Administrators were asked to identify patterns in data (See Attachment A) and then to 
generate three explanations for each pattern. It was stated that the explanations should be 
consistent with the actual data present; the key explanation which articulated what was believed 
to be the root cause of the problem was then linked to the priority in the school and incorporated 
into the school site plan. In other words, administrators were told that school priorities should be 
the best explanations for patterns in data (See Attachment B). 



Measurements , 

This training model has proven to be a major breakthrough in the usefulness ot me 

accountability reports. Because of this training and the feedback provided by administrators, the 
department was able to improve the usability and accuracy of their reports. In analyzing the 
effectiveness of the data planning training model the following measures were used: 

Data Planning Portfolio Scores: The ability to thoughtfully use data in developing 
district and site plans is a key objective under district Goal 1; “to improve teaching and learning 
to enhance the academic achievement of all students.” By 1 996-97, through the school portfolio, 
the District began to measure the process school administrators were using to analyze and plm 
with data. A school was rated exemplary on a “data planning” objective if they demonstrated the 
use of multiple measures for data analysis, sharing of data with the school community, followed 
by reflection and determination of school site priorities based on data. Administrators were 
trained regarding the rubric that was used to measure performance on this objective. 

As part of the portfolio process, school administrators were asked to gather, analyze and 
employ a variety of relative data for improved student achievement and program development. 
Central office administrators, site administrators and teachers were trained to score the objective 
in the portfolio. The process for paired scoring the portfolios was modeled the Bay Area c oo 
Reform Collaborative’s model for school portfolios. The rubric (Attachment C) desire o 
measure the objectives classified schools on a scale of 1 to 9. Verbal descnptors used to 
categorize this variable were as follows: exemplary (score of 7, 8 or 9), satisfactory (score of , 
or 6), and limited (score of 1, 2 or 3) evidence. 
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Professional Development Scores. This objective is also measured through the school 
portfolio. In order to be exemplary on this objective the school needs to provide clear evidence 
of staff participation in planning and implementing professional development activities at the 
school site. The professional development activities clearly focus on the impact on smdent 
achievement. Professional development addresses teaching strategies that meet the differentiated 
learning needs of the smdent population. Individual plans and records are maintained to show 
professional development for teachers. Reflection shows connection and impact on teaching 
practices and smdent learning. 

CTBS Scores-. The Comprehensive Tests of Basic Skills is the norm-referenced 
standardized test administered every year to Grades 2-11. The two sub-tests administered to 
smdents were Reading Comprehension and Mathematics Concepts and Applications. 

Survey Questions: Middle school principals were asked to answer two basic questions; 

• Has planning with data helped you to focus your resources more efficiently? If yes, could 
you briefly describe how? 

• In your best judgement, to what extent has planning with data lead to academic achievement 
at your school? 

Results of Data Planning Training Model 

It was the hope of the Planning, Research, and Evaluation Department that due to the 
extensive data-planning workshops provided for three years to administrators, by 1997-98 the 
school's site plans would be a reflection of priorities that have resulted from data analysis. Of 
the 17 schools, that were measured on the objective stated as “to demonstrate the thoughtful use 
of data in developing district and site plans”, a majority showed improvement. 



Tablel: Data Planning Portfolio Scores for Middle Schools 



Rubric: 


No. of Schools in 1996/97 


No. of Schools in 1997/98 


Exemplary Evidence 


4 


9 


Satisfactory Evidence 


3 


2 


Limited Evidence 


10 


5 


No Evidence 


0 


1 



A majority of the middle schools scored in the exemplary category on this objecUve in 
1997-98 as compared to 1996-97 when the majority of school administrators provided limited 
evidence towards this objective, which is an indication that the workshop was a successful step 
towards the articulation of data. 
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Overall student performance at the middle school level showed continuous improvement 
in both Reading and Math for the 1996-97 and 1997-98 school years (see Chart 1 and 2). The 
percentage of students testing in the bonom quartile decreased. Theses statistics are particularly 
encouraging as students in the bottom quanile are the most academically at risk students and 
were now being addressed due to the disaggregation of data by quartiles. Another group that 
received attention due to the disaggregation of data were the African American and Latino 
students. 




Middle School District wide 
Reading Performance 4-year Overview* 



601 ^ 




(N-9.936) (N-9.788) (N-9.845) (N-9.649) 




Middle School District wide 



Math Performance 4-year Overview 





In order to further understand the impact of the data-planning model on overall academic 
growth, Pearson product-moment correlation coefficient was computed between scores on the 
data-planning portfolio scores and test score gains from the CTBS testing. The gain was 
computed as the difference between the Spring 1997 and Spring 1998 CTBS test. The results are 

reported in Table 2: 



ERIC 



School 

Level 


cianuM 

Number of 
Schools 


Correlation between 
Reading & DP Scores 


P 

value 


Correlation between 
Math & DP Scores 


P 

value 


Elementary 


63 


-.085 


.51 


-.107 ' 


.40 


Middle 


16 


.423 


.10 


.368 


.16 


High 


16 


.127 


.64 


-.072 


.79 



As seen in Table 2, there was a moderate correlation between test score gains and 
portfolio scores on the data planning objective for middle schools — schools with higher data- 
plaiuiing portfolio scores tended to have higher test score gains. There was a zero correlation 
between the two variables at the other two school levels. None of the above correlations were 

significant at the .05 alpha level of significance. 

Another interesting finding was that there is a significant correlation between scores on 
the data planning objective of the school portfolio and on the professional development objective 
of the school portfolio (r = .71, p<.05, n=l03). An inference that can be drawn is that schools 
that obtained high scores on the data-planning objective were simultaneously workmg with then 
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staff on developing data patterns, explanations for those patterns and reform strategies that would 
impact classroom instruction. 

In order to further understand the ways in which the data-planning model has been used 
by administrators to improve the development of school site plans and, in turn, academic 
achievement, middle school principals were asked to answer two basic questions: 

1. Has planning with data helped you to focus your resources more efficiently? If yes. could 
you briefly describe how? 

2. In your best judgement, to what extent has planning with data lead to academic achievement 
at your school? 

Of the 1 7 surveys sent out to principals, 1 1 were returned with responses. All 1 1 middle school 
principals believed that planning with data has helped them to focus their resources. Most 
were also able to articulate steps they took in this process— some paid particular attention to 
skills that were emphasized in the workshops such as the importance of disaggregation. For 
instance, in response to the question asking for a description of how planning with data has 
helped to focus resources more efficiently, one principal stated the following. 

“teachers were able to identify grade level strengths and weaknesses as well as school 
wide strengths and weaknesses.” 

Another principal, also having understood the importance of disaggregating the data stated: 

“we analyze data in order to determine where we will focus our resources and energies 
during the year. We look at the data from the school portfolio and then we review die 
Hqt a from the [Academic Achievement] Volume 3 with an emphasis on the academic 
achievement of the African-American and Latino students. Then as a staff we list how 
we will address our areas of weakness.” 

At the same time, principals were unable to provide clear “cause and effect accounts of 
how data planning had effected academic achievement. Two of the principals, in response to 
how data-planning has lead to achievement at their school, described the achievement itself: 

“Algebra readiness scores have improved, CTBS improved, suspensions are down, etc. 
“Tutoring for bottom quartile over the last 4 years has shown that that quartile is 
becoming significantly smaller each year.” 

Although a cause and effect connection between data-planning and achievement was not 
provided, these two examples, we argue, are evidence that principals are aware of different 
indicators of student improvement and are aware of how to find, read, and interpret the data. 

Similarly, in response to the 2”^ question, eight of the 1 1 principals also did not provi e a 
connection between how data-planning was related to achievement, but rather described data- 
planning as being helpful for allocating resources, for example: 

“By identifying our needs more clearly, our decision making process was enhanced and 

our resources can be expended in a more targeted manner.” 

“Data has provided an impetus for conversation and discussion with the school’s 
stakeholders: teachers, parents, students. It has also provided information as to where we 
need to focus our attention.” 

Only one principal was able to clearly articulate a connection between data-planning and 

achievement. He states, , , • i j 

“our school was reconstituted in 1994... the decision on how cumculum design would 
best meet the needs of the students was based on data and research. With this philosophy 
on decision-making, the school is now matching [similar schools] in academic 
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achievement. The significant gains in academic achievement would not have been 

possible without the use of data.” 

Overall, results of the survey suggest that administrators are actively using data to plan 
and improve instruction. They seem to understand what types of indicators to look for when 
assessing achievement and improvements at their school sites and how to use and read the 
accountability reports to find relevant information. 

Discussion and Implications 

Several benefits have emerged from the data-planning model: ... 

• The delineation of the data-planning model has been a strong educational tool in training 

administrators to read, understand, interpret and discuss data. 

• Administrators both in and outside the District now have knowledge on the process and data 
considered when planning, implementing, and evaluating their site plans. 

• Using a systematic approach to analyzing data has assisted school reform by highlighting the 
strengths and weaknesses in educational interventions. 

• The overall level of concern with accountability and meeting the central objectives of the 

District is increasing. . 

• The District is benefiting from having a common framework for viewing data. This is 
allowing administrators to make more informed decisions and plan using data. 

• Analysis of data has assisted in the communication between administrators and teachers and 
has helped focus the professional development of teachers. 

Extension of the data -planning model among elementary school level administrators was 
conducted in 1998-99. Further, a whole workshop to improve instruction with cntenon 
referenced scores fi-om a standardized test was designed for teachers. This assisted in extending 
the model to teachers and their use of data. Teachers were asked to study the summary data on 
the criterion-referenced scores of the SAT-9. On identifying the content clusters that needed 
attention, teachers were provided with the “Reviewer’s Edition”, a guide for planning classroom 
instruction printed by the publisher. Teachers were asked to identify linguistic and conceptual 
challenges towards the standard or content cluster that needed attention. 

In discussing some of the findings of this study, it was seen that there is a difference 
between the school levels in their correlations between their data planning portfolio scores and 
test score gtiirs. This difference can be attributable to differences between interpretive 
frameworks of elementary and middle school principals. Glasman (1984) found that middle 
school principals on the whole used test score data in decision making more than did elementary 

school principals. ^ ♦ 

The responses received to the survey question, “to what extent has planmng wath data 

lead to academic achievement,” followed a similar pattern to that of a study conducted by Penuel 
et. al. (1998) when principals were asked to describe reasons (or provide data) for why their 
school’s test had increased or declined, most were unsuccessful in matching reasons to 
conclusions. In fact, many principals did not distinguish between cause and effect at all, or 

attributed a decline in test scores to positive programs in the school. 

Limitations: Technical limitations to the current study are the small and unequal sample 
sizes and the external validity of the study. The study describes the process and findings in a 
District and is not generalizable to other Districts. Limitations in the delineation of the data- 
planning process involved gaps in the articulation and delivery of the model with school statt. 
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and the role that analyzing data had in the determination of school site priorities. Alkin and 
Stecher (1983) found that tests and evaluation data were not among the top four sources of 
information: most decision makers relied on beliefs and opinions, program requirements and 
budgets and to a lesser extent direct obsers-ation and parent input (see also Glasman, 1984). 

Next Steps: In order to fill in the gaps regarding the process by which administrators 
reason with and use data, the two attachments A «&. B have become part of every school site plan 
and will be analyzed for all schools. Further, the school portfolio process will be replaced by 
interviews with principals with regards to the following questions: What types of data has your 
school analyzed and used? How is it used? What are your school priorities and how are they 
related to your school’s analysis of data? How have your priorities changed over the past three 
years and why? Show some examples of ways staff use data to improve teaching and learning 
school-wide. Therefore, the data-planning process will involve interview protocols for use with 
principals and videotape planning sessions among principals and between principals and teachers 
as they go through the process of identifying patterns and setting school site priorities. 

Future research by our research team will not only broaden the scope of the study but will 
also involve collecting more ethnographic information on how principals reason in context. 
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aitachment c 

General Topics Exemplary Evidence 



I. 

Multiple 
Measures, 
Variety of Data 



[participation in 
Collection 
and Analysis 
i oi.Data 



a A. Data analysis 
addresses alternative 
assessment tasks 
including Math 
performance tasks 

□ B. Connections are made 
between two or more types 
of data. 

□ C. Evidence includes 
summaries and analysis. 

o D. Pre-post, long-term, 
data analysis is included. 

D E. Analysis is related to 
targeted student groups 
such as AA. U ELL GATE 



□ A. Evidence shows staff 
participation in analysis of 
school and individual 
student data. 

B B. Evidence shows parent 
and/or student 
partictpation in analysis of 
data. 

B C. staff uses analysis of 
• data to shape and modify 
programs and strategies 
to meet student needs. 



Satisfactory Evidence 

B A. Data and analysis include 
disagregated standardized 
test information. 

B B. Variety of data includes 
at least one of the following: 
observations, surveys, 
grades, records. 

B C. Evidence includes data 
description, reports, tools. 

B D. Longitudinal data 
information is included. 

B E. Data shown is related to 
targeted groups including all 
quartiles 



B A. Ev’ lence shows staff 
participation in collection of 
school data. 

B B. Evidence of others 
involved in data collection 
including parents and 
students. 

B C. Process for analysis of 
student data is shown 






Limited Evidence 

B A. Limited evidence of data 
analysis of disagregated 
standardized test 
information. 

B B, A variety of data types is 
no in evidence. 

B C. Evidence may not include 
actual instruments used or 
summaries/ results. 

B D. Limited evidence of 
longitudinal data being used. 

B E. Data is not specifically 
focused on target student 
populations. 



a A. Limited evidence of staff 
involvement in collaboration 
and analysis of data 
D B. Limited evidence of 
involvement of other 
stakeholders in data 
collection and analysis, 
a C. Process of analyzing data 
is not evident 



III. 

Alignment 
of Data with 
District 
Obiectives 



■r 



^ IV. 

1 Reflection- 
'Making . 
Connections 



% 
ERIC 



a A. Analysis of data is 
aligned with content areas 
and site activities. 

D B. Connections are 
shown through data 
analysis aligned with 
school program activities. 

D C. Evidence shows 
analysis of student work/ 
learning. 

a A Parents use data to 
make plans and decisions, 
o B. Teachers use data to 
adjust classroom 
strategies and 
instnictionai practices, 
o C. Staff shows evidertce 
of analysis and reflective 
practices in use of data to 
make decisions related to 
school programs. 



o A. Analysis of data is aligned 
with site priorities, 
o B. Data results and analysis 
match activities in site plan, 
a C. Data analysis focuses on 
student teaming. 



a A. Use of data is reffected in 
parent and community 
activities. 

a B. School programs and 
classroom planning may be 
modified based on analysis 
of data. 

a C. Evidence of reflection 
based on data and results of 
- - analysis. 

‘zO 



B A. Evidence of data 
connected to site priorities 
not clearly shown. 

B B. Limited evidence of data 
artaiysis connected to site 
plan activities. 

a C. Limited evidence of data 
connected to student work. 



Q A. Limited evidence of parent 
involvement in use of data, 
o B. School programs and 
classroom activities show no 
revision or modification based 
on analysis of data, 
o C. Limited evidence of 
school reflection on data and 
analysis 
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